1. Field of the Invention
The present invention relates to a pattern-matching processing method and an image processing apparatus. More particularly, the present invention relates to a pattern-matching processing method and an image processing apparatus that relates to a distance measuring system adopting image processing, and in which pattern matching is performed by generating left and right matching images to which pixel interpolation has been introduced from the photographed original images of an object taken by a stereo camera.
2. Description of the Related Art
Generally, the so-called stereoscopic distance measuring method is widely known as a three-dimensional measuring technique adopting image processing, in which correlation of a pair of left and right images of an object photographed from different positions by a stereo camera composed of two cameras is found and the distance is found from the parallax of the same object using image processing based on the principle of the triangulation using the camera parameters such as the position of the stereo camera and the focal length.
Such a distance measuring method is applied to, for example, distance measurement of an object located ahead of a vehicle such as a car. A stereo camera facing ahead is mounted in a vehicle and the distance to an object located ahead of the vehicle is measured based on the photographed stereoscopic images of the object ahead of the vehicle.
The distance information of an object calculated based on left and right images photographed by a stereo camera is sent to, for example, a vehicle control unit or a warning device, and is used to, for example, immediately stop the vehicle according to the measured distance or to inform the driver of the distance to the vehicle in the road ahead.
When such a stereoscopic distance measuring system is mounted in a vehicle, the system is usually equipped with a pattern-matching processing section, a distance measuring section, and an object recognition processing section for left and right images taken by a stereo camera.
In the pattern-matching processing section, left and right pattern-matching images in a small area are generated from the left and right original images taken by the stereo camera, and stereoscopic pattern matching is performed to find correlation between each other based on the left and right pattern-matching images.
Based on the matching results obtained by the stereoscopic pattern-matching processing, the difference between positions of pixels in the left and right original images (that is, the parallax) relevant to the object photographed is measured, and the distance from the stereo camera to the object is calculated using the measured parallax and the parameters relating to the camera.
Based on the generated distance data, whether the object photographed in the original image is, for example, a vehicle in the road ahead is judged. The result of the judgment is sent to a vehicle control unit that performs the vehicle traveling control, and so on, in order to support driving or to avoid danger.
By the way, the pattern-matching images generated in the pattern-matching processing section are composed of, for example, nine pixels, that is, a 3×3 pattern consisting of three transverse pixels and three vertical pixels, which are cut out respectively from the left and right original images photographed by the stereo camera.
When pattern matching is performed on these pattern-matching images, if it is assumed that the left original image is, for example, the reference image and the right original image is a comparison image, the right pattern-matching images, which are cut out from the right original image while being shifted pixel by pixel, are compared with the left pattern-matching image cut out from the left original image. At this time, a correlation coefficient of each pattern-matching image is calculated and the degree of matching is judged according to the magnitude of the correlation coefficient.
When it is judged that the left pattern-matching image and the right pattern-matching image match each other, the position of the central pixel in the left pattern-matching image of the left original image and the position of the central pixel in the right pattern-matching image of the right original image are found. The parallax in the transverse direction is found from the difference between the left pixel position and the right pixel position. In addition, the parallax in the vertical direction is also found in the same technique. If the parallax of the object viewed from the stereo camera can be found, it is possible to calculate the distance between the stereo camera and the object by reference to the parameters of the camera.
According to a technique using a technology in which, as described above, the three-dimensional position of an object ahead of a vehicle is measured, a matching position at which left and right pattern-matching images match each other is found by performing stereoscopic matching between the left and right pattern-matching images in small areas extracted from the left and right original images photographed in stereo, and the distance between the positions of pixels of the corresponding left and right pattern-matching images is output as a parallax.
As the parallax obtained by processing the stereoscopic images are expressed in the unit of pixels in the image, if a distance is calculated from the parallax expressed in the unit of pixels by using the triangulation, the resolution is degraded for longer distances to an object.
Because of the degradation of resolution, an error is produced in the calculated distance. This error is ascribed to the fact that the parallax detected from images photographed by a stereo camera is expressed in the unit of pixels and is also a main cause of the degradation of resolution in distance measurement based on the principle of the triangulation. This error commonly occurs in a case where other matching positions in images are found and may affect the judgment of an object in the object recognition section significantly.
Conventionally, as a consequence, resolution is improved to secure the accuracy in distance measurement by interpolating pixels in each left original image and right original image photographed in stereo, respectively, when pattern-matching processing is performed.
There are various known methods as a conventional pixel interpolation techniques. In one of them, pixel interpolation is performed based on eight pixels located adjacently to the pixel at the position to be interpolated. In this method, the pixel values of each of eight pixels adjacently surrounding the position to be interpolated are averaged and the resultant value is taken as a pixel value for the position to be interpolated. Then, for the whole of the original image, pixel interpolation is performed between two adjacent pixels successively.
This interpolation technique requires a tremendous amount of calculation because interpolated images are obtained after the interpolation pixel values to be interpolated are calculated for the whole of the original image. As a result, it takes a long time for the processing of interpolation; therefore, a more or less simplified interpolation technique is also adopted.
In the simplified pixel interpolation technique for obtaining an interpolated image, two adjacent pixels are selected, as basic interpolation processing, and an interpolation pixel is interpolated sequentially between the two adjacent pixels, with the average of the pixel values relating to the two pixels being taken as a pixel value for interpolation.
First, two transversely adjacent pixels in the original image are selected and a pixel is interpolated sequentially between the two adjacent pixels, with the average of the pixel values of the two pixels being taken as a pixel value for interpolation. This process, in which a pixel is interpolated between the two transversely adjacent pixels in the transverse row in the original image, is performed from the top of the original image to its bottom sequentially. After this, pixel interpolation is performed between two vertically adjacent pixels in the original image. By utilizing the interpolation pixels already interpolated between pixels in the transverse direction as well, pixel interpolation with the seven adjacent pixels is performed between the two vertically adjacent pixels in the original image. According to the pixel interpolation technique described above, interpolated images are obtained.
In this simplified interpolation technique described above, which has been adopted to obtain interpolation pixels, for two vertically adjacent pixels in the original image, the average value of the two adjacent pixels is taken as the interpolation pixel value and, as for the vertical direction, the average value of pixel values each of the seven adjacent pixels has is taken as the interpolation pixel value, with the already calculated interpolation pixels being also used, therefore, the amount of calculations required for the pixel interpolation technique is considerably reduced compared to the case where the pixel value of the position to be interpolated is obtained from the average value of eight adjacent pixels.
However, a distance measuring apparatus to be mounted in a vehicle is required to deliver as accurately and speedily as possible the distance measurement data to be supplied for driving support and warning and, therefore, there remains a problem that the amount of calculations is still large and much time is required, if the conventional or simplified pixel interpolation technique is used without changes made thereto for generating a pattern-matching image from a stereoscopic image.
The object of the present invention is, therefore, to provide a pattern-matching processing method and an image processing apparatus, in which pattern matching is performed by introducing pixel interpolation to the left and right pattern-matching images in small areas extracted from the photographed original images of an object taken by a stereo camera.